Controlling input to a plurality of computer windows

ABSTRACT

A method for controlling user input to a plurality of computer windows is provided. The method determines an input window and an output window, directs a plurality of input commands including primary and secondary input commands to the input window, determines when a user is visually focusing on the output window, and directing the secondary input commands to the output window, while the primary input commands remain directed to the input window.

BACKGROUND

The present invention generally relates to controlling input to aplurality of computer windows, and more particularly to directing inputto a computer window that a user is visually focused on.

In a multiple window computing environment, a computer user may havemultiple computer windows open or displayed on a display device (e.g., acomputer monitor). To enter data into a particular computer window(i.e., directing input commands to an application associated with theparticular computer window), the user focuses on the particular windowor causes the particular window to become active. Focusing (oractivation) may occur by some activity by the user, e.g., clicking onthe desired window with a cursor on the display and a mouse button, orby hovering the cursor over the desired window. Generally, suchactivation requires physical manipulation of an input device, e.g., bythe user's hands. If a second computer window is open, the user focuseson (or activates) the second computer window prior to entering data intothe second window. Typically, when focus is directed to a particularwindow, data entry (including input commands) is exclusively directed tothe active window. To resume entering data in the initial computerwindow, the user returns focus to the initial computer window and thesecond computer window becomes inactive.

SUMMARY

According to one embodiment, a method for controlling user input to aplurality of computer windows is provided. The method may includedetermining an input window from the plurality of computer windows,determining an output window from the plurality of computer windows,directing a plurality of input commands to the input widow. Theplurality of input commands includes at least one of one or more primaryinput commands and one or more secondary input commands. The method mayfurther include determining that the user is visually focusing on theoutput window and directing, in response to the determination that theuser is visually focusing on the output window, the one or moresecondary input commands to the output window. The one or more primaryinput commands remain directed to the input window.

According to another embodiment, a computer program product forcontrolling user input to a plurality of computer windows is provided.The computer program product may include at least one computer readablenon-transitory storage medium having computer readable programinstructions for execution by a processor. The computer readable programinstructions include instructions for determining an input window and anoutput window from the plurality of computer windows, directing aplurality of input commands to the input widow including at least one ofprimary and secondary input commands, determining that the user isvisually focusing on the output window, and directing the secondaryinput commands to the output window.

According to another embodiment, a computer system for controlling userinput to a plurality of computer windows is provided. The system mayinclude at least one processor, at least one computer readable memory,at least one computer readable tangible, non-transitory storage medium,and program instructions stored on the at least one computer readabletangible, non-transitory storage medium for execution by the at leastone processor via the at least one computer readable memory. The programinstructions include instructions for determining an input window and anoutput window from the plurality of computer windows, directing aplurality of input commands to the input widow including at least one ofprimary and secondary input commands, determining that the user isvisually focusing on the output window, and directing the secondaryinput commands to the output window.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The following detailed description, given by way of example and notintended to limit the invention solely thereto, will best be appreciatedin conjunction with the accompanying drawings, in which:

FIG. 1 is a diagram illustrating a system employing a method forcontrolling user input to a plurality of computer windows, according toan embodiment;

FIG. 2 is a flowchart illustrating a method for controlling user inputto a plurality of computer windows, according to an embodiment; and

FIG. 3 is a block diagram illustrating a computing node, according to anaspect of the invention.

The drawings are not necessarily to scale. The drawings are merelyschematic representations, not intended to portray specific parametersof the invention. The drawings are intended to depict only typicalembodiments of the invention. In the drawings, like numbering representslike elements.

DETAILED DESCRIPTION

Various embodiments of the present invention will now be discussed withreference to FIGS. 1-3, like numerals being used for like andcorresponding parts of the various drawings.

According to one embodiment, a method is provided for controlling userinput to a plurality of computer windows by determining an input windowand an output window, directing input commands including at least one ofprimary and secondary input commands to the input window, determiningwhen the user is visually focusing on the output window, and directingthe secondary input commands to the output window.

FIG. 1 depicts a diagram illustrating an exemplary system 1000 employinga method for controlling user input to a plurality of computer windows,according to an embodiment. System 1000 includes a display 100, inputmeans (e.g., keyboard 121 and mouse 122), computer 130, and a videoimaging device 140 (e.g., a webcam). Display 100 can display a pluralityof computer windows, e.g., a first computer window 101 and a secondcomputer window 102. A user 110 may direct the user's sightline 111(i.e., the direction of the user's sight) toward one of the computerwindows, e.g., first computer window 101. User 110 may provide input tocomputer 130, and in particular, to a program/application/fileassociated with an active computer window (e.g., 101, 102) displayed ondisplay 100. For example, first computer window 101 may be an activecomputer window and input from keyboard 121 and/or mouse 122 may bedirected to a program on computer 130 associated with first computerwindow 101. The user's sightline 111 may be determined from videoimagery captured with video imaging device 140.

The user's sightline may be determined using any known or contemplatedsight recognition (or gaze detection) technique or method. For example,determining that the user is visually focusing on a particular computerwindow may be done according to the methods disclosed in InternationalPublication No. WO 2012/008827 A1, the contents of which areincorporated herein by reference.

In one embodiment, an input window and an output window are determinedfrom the plurality of computer windows on display 100. For example, aninput window may be a computer window associated with a word processingprogram and an output window may be a computer window associated with areference document (such as a webpage, a PDF, a separate word processingdocument, etc.). The input window and the output window may beassociated with the same program/application, e.g., a first wordprocessing document and a second word processing document, or the inputwindow and the output window may be associated with differentprograms/applications, e.g., a word processing application and aspreadsheet application. All user input may be directed to the inputwindow (i.e., to the program/application associated with the inputwindow). User input, as contemplated herein, may include two types ofinput commands: primary input commands and secondary input commands.Primary input commands can include, but are not limited to, keystrokesfrom keyboard 121 and/or motion and button clicks from mouse 122.Secondary input commands can include one or more types of inputcommands, such as a scrolling command. The primary and secondary inputcommands may be separate and exclusive of each other. For example,primary input commands may be any input command excluding apredetermined secondary input command, such as a scrolling command. Upona determination that the user is visually focusing on the output window,the secondary input commands may be directed to the output window.Primary input commands may continue to be directed to the input window.Upon a determination that the user is visually focusing on the inputwindow, the secondary input commands may be directed to the outputwindow. The determinations of the user's visual focus may be performedwith visual imagery of the user (e.g., by detecting the user'ssightline), and may be continuous. In other words, the user's sightlinemay be continuously monitored to determine when the user is visuallyfocusing on an input window or an output window.

FIG. 2 is a flowchart illustrating an exemplary method 200 forcontrolling user input to a plurality of computer windows, according toan embodiment. At 201, an application associated with the disclosedmethod is loaded on a computer. The application may engage a visualimaging device connected to the computer. The application may designatetypes of input commands as primary input commands and secondary inputcommands according to predetermined rules or settings. In oneembodiment, the primary input commands may include any input commandexcluding a predetermined selection of other input commands, e.g., thesecondary input commands. In one embodiment, the primary input commandsmay include any input command excluding a scrolling command provided bya scroll wheel housed in the peripheral mouse and the secondary inputcommand may include the scrolling command provided by a scroll wheelhoused in a peripheral mouse. In one embodiment, the scrolling commandmay be provided by a touchpad, e.g., by a swiping gesture.

At 202, an input window and an output window is determined from aplurality of computer windows displayed on a display monitor, e.g.,connected to the computer running the application loaded in 201. Theinput window may be an active computer window, which a user may haveactivated by some activity, e.g., by selecting the computer window witha cursor. The output window may be an inactive computer window.

At 203, the primary input commands and secondary input commands aredirected to the input window. At 204, the user's sightline is detectedand it is determined whether the user is visually focused on (i.e.,looking at) the output window. If the user is visually focused on theoutput window, at 205, the secondary input commands are directed to theoutput window. If the user is not visually focused on the output window,the secondary input commands are directed to the input window. Theprimary input commands may remain directed to the input window when thesecondary input command are directed to the output window. Thedetermination of the user's sightline (i.e., determining when the useris visually focused on the output window) may be continuously performed.

The directing of the input commands (i.e., the primary input commands,the secondary input commands) to the desired computer windows may beachieved by any method or technique known to or contemplated by thoseskilled the art. For example, directing the input commands may occur bysending input events over a message bus.

In some embodiments, the determining whether the user is visuallyfocusing on an output window may include detecting that the user'ssightline is directed to an area on a display associated with the outputwindow displayed on the display. The determining whether the user isvisually focusing on an output window may include determining an amountof time that the user's sightline is directed to the output window. Athreshold amount of time that the user's sightline is directed to theoutput window may be preconfigured, and upon meeting the thresholdamount of time it may be determined that the user is visually focusingon the output window.

In an exemplary scenario, according to an embodiment, a user may havetwo computer windows open on a display monitor, with the first windowbeing associated with a word processing application and the secondwindow being associated with a reference document (e.g., a webpage in aweb browser). The first window may be designated as the input window andthe second window may be designated as the output window. The user mayenter text (i.e., primary input commands) into the first window (i.e.,input window) and when the user looks at the second window the user mayscroll (i.e., secondary input command) through the document displayed inthe second window (i.e., output window). During the time when the useris looking at the second window, the user may continue to enter textinto the document open in the first window. The user may toggledirecting the secondary input commands (between the input and outputwindows) without physically manipulating an input device to otherwiseselect the desired window.

According to one embodiment, two active focus areas are simultaneouslymaintained within a graphic user interface on a display monitor with oneactive focus area designated as an input focus area and the secondactive focus area designated as an output focus area. Input commands aredirected to the active focus areas with the primary input commands beingdirected to the input focus area and the secondary input commands beingdirected to the output focus area. In one embodiment, the secondaryinput commands are directed to the output focus area in response to adetermination that a user is visually focusing on the output focus area.In another embodiment, the secondary input commands are directed to theinput focus area in response to a determination that the user isvisually focusing on the input focus area.

In an embodiment, a method disclosed herein further includes determiningthat the user is visually focusing on the input window and, in responseto that determination, directing the secondary input commands to theinput window.

The secondary input commands may include any type of input command,e.g., scrolling, page up, page down, etc. In an embodiment, thesecondary input commands are a small selection of various types of inputcommands. In an embodiment, the secondary input commands are single typeof input command. In an embodiment, the secondary input commands includescrolling commands. In another embodiment, the secondary input commandsare limited to scrolling commands. In an embodiment, the primary inputcommand include input commands provided by a keyboard.

The embodiments disclosed herein may include multiple output windows.The embodiments disclosed herein may include multiple display monitors.In some embodiments, an input window may be displayed on a first monitorand the output window may be displayed on a second monitor.

In an embodiment, a method disclosed herein further includes determiningone or more additional output windows from a plurality of computerwindows, determining that the user is visually focusing on one of theadditional output windows, and directing the secondary input commands tothat additional output window.

FIG. 3 depicts a schematic illustrating an example of a computing node.Computing node 10 is only one example of a suitable computing node andis not intended to suggest any limitation as to the scope of use orfunctionality of embodiments of the invention described herein.Regardless, computing node 10 is capable of being implemented and/orperforming any of the functionality set forth hereinabove.

In computing node 10 there is a computer system/server 12, which isoperational with numerous other general purpose or special purposecomputing system environments or configurations. Examples of well-knowncomputing systems, environments, and/or configurations that may besuitable for use with computer system/server 12 include, but are notlimited to, personal computer systems, server computer systems, thinclients, thick clients, hand-held or laptop devices, multiprocessorsystems, microprocessor-based systems, set top boxes, programmableconsumer electronics, network PCs, minicomputer systems, mainframecomputer systems, and distributed computing environments that includeany of the above systems or devices, and the like.

Computer system/server 12 may be described in the general context ofcomputer system-executable instructions, such as program modules, beingexecuted by a computer system. Generally, program modules may includeroutines, programs, objects, components, logic, data structures, and soon that perform particular tasks or implement particular abstract datatypes. Computer system/server 12 may be practiced in distributedcomputing environments where tasks are performed by remote processingdevices that are linked through a communications network. In adistributed computing environment, program modules may be located inboth local and remote computer system storage media including memorystorage devices.

As shown in FIG. 3, computer system/server 12 in computing node 10 isshown in the form of a general-purpose computing device. The componentsof computer system/server 12 may include, but are not limited to, one ormore processors or processing units 16, a system memory 28, and a bus 18that couples various system components including system memory 28 toprocessor 16.

Bus 18 represents one or more of any of several types of bus structures,including a memory bus or memory controller, a peripheral bus, anaccelerated graphics port, and a processor or local bus using any of avariety of bus architectures. By way of example, and not limitation,such architectures include Industry Standard Architecture (ISA) bus,Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, VideoElectronics Standards Association (VESA) local bus, and PeripheralComponent Interconnects (PCI) bus.

Computer system/server 12 typically includes a variety of computersystem readable media. Such media may be any available media that isaccessible by computer system/server 12, and it includes both volatileand non-volatile media, removable and non-removable media.

System memory 28 can include computer system readable media in the formof volatile memory, such as random access memory (RAM) 30 and/or cachememory 32. Computer system/server 12 may further include otherremovable/non-removable, volatile/non-volatile computer system storagemedia. By way of example only, storage system 34 can be provided forreading from and writing to a non-removable, non-volatile magnetic media(not shown and typically called a “hard drive”). Although not shown, amagnetic disk drive for reading from and writing to a removable,non-volatile magnetic disk (e.g., a “floppy disk”), and an optical diskdrive for reading from or writing to a removable, non-volatile opticaldisk such as a CD-ROM, DVD-ROM or other optical media can be provided.In such instances, each can be connected to bus 18 by one or more datamedia interfaces. As will be further depicted and described below,memory 28 may include at least one program product having a set (e.g.,at least one) of program modules that are configured to carry out thefunctions of embodiments of the invention.

Program/utility 40, having a set (at least one) of program modules 42,may be stored in memory 28 by way of example, and not limitation, aswell as an operating system, one or more application programs, otherprogram modules, and program data. Each of the operating system, one ormore application programs, other program modules, and program data orsome combination thereof, may include an implementation of a networkingenvironment. Program modules 42 generally carry out the functions and/ormethodologies of embodiments of the invention as described herein.

Computer system/server 12 may also communicate with one or more externaldevices 14 such as a keyboard, a pointing device, a display 24, etc.;one or more devices that enable a user to interact with computersystem/server 12; and/or any devices (e.g., network card, modem, etc.)that enable computer system/server 12 to communicate with one or moreother computing devices. Such communication can occur via Input/Output(I/O) interfaces 22. Still yet, computer system/server 12 cancommunicate with one or more networks such as a local area network(LAN), a general wide area network (WAN), and/or a public network (e.g.,the Internet) via network adapter 20. As depicted, network adapter 20communicates with the other components of computer system/server 12 viabus 18. It should be understood that although not shown, other hardwareand/or software components could be used in conjunction with computersystem/server 12. Examples, include, but are not limited to: microcode,device drivers, redundant processing units, external disk drive arrays,RAID systems, tape drives, and data archival storage systems, etc.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

Detailed embodiments of the claimed structures and methods are disclosedherein; however, it can be understood that the disclosed embodiments aremerely illustrative of the claimed structures and methods that may beembodied in various forms. This invention may, however, be embodied inmany different forms and should not be construed as limited to theexemplary embodiments set forth herein. Rather, these exemplaryembodiments are provided so that this disclosure will be thorough andcomplete and will fully convey the scope of this invention to thoseskilled in the art. In the description, details of well-known featuresand techniques may be omitted to avoid unnecessarily obscuring thepresented embodiments.

References in the specification to “one embodiment”, “an embodiment”,“an example embodiment”, etc., indicate that the embodiment describedmay include a particular feature, structure, or characteristic, butevery embodiment may not necessarily include the particular feature,structure, or characteristic. Moreover, such phrases are not necessarilyreferring to the same embodiment. Further, when a particular feature,structure, or characteristic is described in connection with anembodiment, it is submitted that it is within the knowledge of oneskilled in the art to affect such feature, structure, or characteristicin connection with other embodiments whether or not explicitlydescribed.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the invention.The terminology used herein was chosen to best explain the principles ofthe embodiment, the practical application or technical improvement overtechnologies found in the marketplace, or to enable others of ordinaryskill in the art to understand the embodiments disclosed herein.

What is claimed is:
 1. A method for controlling user input to aplurality of computer windows, the method comprising: determining aninput window from the plurality of computer windows; determining anoutput window from the plurality of computer windows; directing aplurality of input commands to the input window, wherein the pluralityof input commands includes at least one of one or more primary inputcommands and one or more secondary input commands; determining that theuser is visually focusing on the output window; and directing, inresponse to the determination that the user is visually focusing on theoutput window, the one or more secondary input commands to the outputwindow, and wherein the one or more primary input commands remaindirected to the input window.
 2. The method according to claim 1,further comprising: determining that the user is visually focusing onthe input window; and directing, in response to the determination thatthe user is visually focusing on the input window, the one or moresecondary input commands to the input window.
 3. The method according toclaim 1, wherein the one or more secondary input commands includescrolling commands.
 4. The method according to claim 3, wherein the oneor more secondary input commands are limited to the scrolling commands.5. The method according to claim 1, wherein the one or more primaryinput commands include input commands provided by a keyboard.
 6. Themethod according to claim 1, wherein the determining when the user isvisually focusing on the output window comprises: capturing video imagesof the user with a video imaging device; and determining, from the videoimages, a direction of the user's sight.
 7. The method according toclaim 1, further comprising: determining one or more additional outputwindows from the plurality of computer windows; determining that theuser is visually focusing on one of the one or more additional outputwindows; and directing, in response to the determination that the useris visually focusing on the one of the one or more additional outputwindows, the one or more secondary input commands to the one of the oneor more additional output windows.